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Applications Quest: A Case 
Study on Holistic Diversity 
in Admission 


Abstract 

After the 2003 U.S. Supreme Court decisions on the University of Michigan admission cases, which 
struck down racial preferences and quotas in Michigan’s undergraduate and law school admission, several 
groups have challenged race-conscious admission, school placement policies and academic support pro- 
grams. Even the federal government has challenged race-conscious scholarship and support programs at 
various institutions. Race-conscious affirmative action policies in higher education have been in existence 
since the early 1960s. At a time when there is significant momentum to attack these programs, solutions 
that adhere to the 2003 U.S. Supreme Court decisions that include race, gender and national origin are 
rare or essentially non-existent. One solution to this 30 year-old debate comes in the form of software that 
allows for holistic comparison of applications in the admission process. 


This paper will focus on one particular program, Applica- 
tions Quest, and discuss a recent case study using its appli- 
cation with real admission data. The conclusion will provide 
implications of software programs, like Applications Quest, 
on race-conscious admission, school placement and aca- 
demic support policies and offer recommendations for future 
research. 

Introduction 

As a result of the 2003 U.S. Supreme Court decisions in 
Gratz v. Bollinger (2003) and Grutter i/. Bollinger (2003), 
the "university campus today is the site of much backlash’’ 
(Chubin & Malcom, 2006). Recall that the Gratz v. Bollinger 
decision struck down the race based point system in under- 
graduate admission at Michigan and the Grutter v. Bollinger 
decision upheld the “narrowly tailored” use of race in admis- 
sion. This backlash can be captured in a letter to the Chicago 
Sun-Times- 


As many predicted after the 2003 U.S. Supreme Court 
decisions on the University of Michigan admissions cases, 
the ambiguities in those decisions and the absence of de- 
finitive guidance from the U.S. Departments of Education 
and Justice have encouraged activist groups to challenge 
universities on the use of race in the conduct of admis- 
sions, financial aid, and academic support programs. Some 
colleges and universities have capitulated in the face of 
threats of legal action and intimidation. Many have volun- 
tarily scrapped programs designed to serve underrepresent- 
ed minority students for fear that they would become tar- 
gets. Worse still, the federal government is complicit in this 
activity by challenging minority focused scholarship and 
support programs at several institutions and threatening to 
withhold federal funding for research and education.... It 
is disappointing that many universities have not stood their 
ground and, instead, have succumbed to risk-averse legal 
advice that suggests that it is better to switch than to fight. 
(Slaughter, 2006) 
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An anti-affirmative action organization has estimated 
that more than 100 colleges “have voluntarily abandoned 
race restrictions [as a criterion], and only a handful have re- 
fused to do so” (Schmidt, 2006) in undergraduate admis- 
sion. Several race-specific programs have fallen to this same 
backlash (Schmidt, 2004). On a state-by-state basis, various 
organizations have vowed to attack race-conscious policies. 
As a result, California has passed Proposition 209 (Myers, 
2005) and Michigan recently passed Proposal 2 (Michigan 
Proposal 2, 2006). Both prohibit preferential treatment on 
the basis of race, gender or ethnicity. Research has shown 
that there are educational benefits to diversity (Chang, et. al. 
1999; Maruyama & Moreno, 2000; Gurin et. al. 2002; Marin, 
2000; Moses & Chang, 2006). President George W. Bush 
also supports diversity but acknowledges the challenges uni- 
versities are facing: 

I strongly support diversity of all kinds, including racial di- 
versity in higher education. But the method used by the 
University of Michigan to achieve this important goal is 
fundamentally flawed. America is a diverse country, racially, 
economically, and ethnically. And our institutions of higher 
education should reflect our diversity. A college education 
should teach respect and understanding and goodwill. And 
these values are strengthened when students live and learn 
with people from many backgrounds. Yet quota systems 
that use race to include or exclude people from higher edu- 
cation and the opportunities it offers are divisive, unfair 
and impossible to square with the Constitution. (President 
Bush discusses affirmative action case, 2003) 

Based on these findings, why is there still a debate on 
this issue? Why didn’t the 2003 Supreme Court decisions 
put an end to this debate? In 2003, the U.S. Supreme Court 
struck down racial quotas to achieve diversity and left the 
decision to diversify up to institutions (Malcom et. al. 2004). 
These decisions also left the door open for more debate on 
how to achieve diversity. 

The U.S. Supreme Court recently heard two cases on 
race-conscious school placement policies (Parents Involved in 
Community Schools v. Seattle School District No. 551, 2007; 
Meredith v. Jefferson County Board of Education, 2006). A 
Seattle (WA) parents group and a Kentucky parent challenged 
the use of race in school placements within their communi- 
ties. In the Seattle case, the court considered whether Seat- 
tle’s so-called integration tiebreaker system, which had been 
discontinued, was tailored to meet a “compelling interest” by 
the school. In Kentucky, Crystal Meredith, a mother of a Lou- 
isville (KY) child, claimed her son was denied entrance into 


the neighborhood school because he is white. In June 2007, 
the Supreme Court struck down both school systems’ policies 
that used racial classifications to determine which schools 
students could attend. In these hearings, Justice Antonin 
Scalia asked, “Is there anything unconstitutional about de- 
signing a mingling of the races and having policies to achieve 
a racial mix?” (Asquith, 2006) Furthermore, Justice Ruth 
Bader Ginsburg said, “It’s hard for me to see how you have a 
racial objective but a nonracial means to get there. 

How would you get there without quotas?” (Asquith, 
2006) The answer to this question is Applications Quest and 
other holistic review software programs. 


If a holistic review is implemented such that 
it doesn’t give preferential treatment to race, 
gender or national origin then this process 
should be reproducible and measurable. A 
reproducible holistic review process will 
yield the same results when given the same 
applications. This does not occur in manual 
holistic review processes. 


The Applications Quest Model 

After the 2003 U.S. Supreme Court decisions on the Uni- 
versity of Michigan admission cases, Applications Quest was 
created (Gilbert, 2006). Applications Quest can best be de- 
scribed as a data mining and analysis software tool that fa- 
cilitates holistic review in admission, school placement and 
academic support programs. The underlying concept behind 
Applications Quest is holistic comparisons of applications. 
In the 2003 U.S. Supreme Court decisions (Gratz v. Bol- 
linger, 2003; Grutter v. Bollinger, 2003), it was clear that 
the only permissible use of race in admission was a holistic 
approach where race was one of many attributes and race 
did not receive preferential treatment. This is also the case 
in Proposition 209 in California (Myers, 2005) and the more 
recent Proposal 2 in Michigan. In fact, Proposal 2 specifically 
bans “programs that give preferential treatment to groups or 
individuals based on their race, gender, color, ethnicity or 
national origin for public employment, education or contract- 
ing purposes.” (Michigan Proposal 2, 2006) The keyword is 
“preferential” treatment. If an admission process were per- 
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forming a true holistic evaluation, then, according to all the 
aforementioned cases, race, gender or national origin could 
be used. The problem that exists today is that no one has 
devised a truly holistic evaluation system that does not give 
preferential treatment to race, gender, national origin, etc., 
such that it can be proved that no preference was given. The 
University of Michigan implemented a holistic review process, 
but its process is more subjective than holistic (University of 
Michigan, 2006). Essentially, its manual, holistic review will 
suffer from critical flaws that prohibit it from being truly ho- 
listic and fair because these approaches are more subjective, 
not reproducible or measurable. 

Reproducible and Measurable 

If a holistic review is implemented such that it doesn’t give 
preferential treatment to race, gender or national origin then 
this process should be reproducible and measurable. A re- 
producible holistic review process will yield the same results 
when given the same applications. This does not occur in 
manual holistic review processes. For example, select an 
admission team at any university that uses manual holistic 
review. Give them 100 applications to review and ask them 
to recommend 25 for admission using their holistic review 
process. Wait one year and then repeat this exercise with the 
same 100 applications. There is no guarantee that the admis- 
sion team will produce the exact same results given the same 
applicant pool; therefore, their process is not reproducible. 
Furthermore, a holistic review process should be measurable. 
When one application is recommended over another applica- 
tion, can that decision be measured in such a way that race, 
gender, national origin are not given preferential treatment? 
If this is not the case, then your holistic review process is not 
measurable. At best, manual holistic reviews are subjective; 
therefore, they are not reproducible or measurable. As a re- 
sult, these subjective reviews can be criticized for being pref- 
erential on the basis that the admission team can not prove 
that they are not. Software like Applications Quest provides a 
reproducible and measurable approach to holistic review. 

Measuring Differences 

In Applications Quest, holistic review is accomplished by 
measuring the difference, or similarity, between applications. 
Imagine if two applications could be compared in such a 
manner that their relative difference, or similarity, could be 
quantified on a 100 point percentage scale. Using this notion, 
two identical applications would be 0 percent different and 
100 percent similar. All other application comparisons would 
fall between 0 percent and 100 percent on this relative scale. 


If this measure could be obtained, then one could compare 
every application to every other application, which would re- 
sult in a difference matrix (see Table 1). 


Table 1. Sample Difference Matrix 


Application ID1 

Application ID2 

Difference 

0 

1 

30% 

0 

2 

50% 

1 

2 

70% 


The difference matrix contains application pairs and their 
measured difference. Using this difference matrix it is easy to 
imagine that some applications would be more similar than 
others. If the difference matrix were mapped onto paper where 
each application was represented by a point, there would be 
clusters of applications (or points), such that the applications 
within a specific cluster are more similar than those outside 
of the cluster. Essentially, these clusters represent holistic, 
diverse applicant pools and can facilitate holistic review. By 
selecting applications from each cluster, holistic diversity 
can be optimized. In an effort to evaluate the effectiveness of 
the Applications Quest model, a case study was done. 

The Case Study 

In a small controlled case study, admission applications for a 
psychology graduate program at a research extensive univer- 
sity were processed in Applications Quest. This psychology 
program received 236 applications in 2004 for 17 admis- 
sion slots. Electronic copies of the 236 applications were 
provided for this study. Seventeen of the 236 applications 
were marked as “Accept”. The others were rejected. The ap- 
plication attributes were ID, application type (US or Foreign), 
citizenship, gender, race, degree (doctoral or master), institu- 
tion of first degree (undergraduate), major at first institution, 
grade point average at the first institution, second institu- 
tion, major at second institution, grade point average at the 
second institution, third institution, major at third institution, 
grade point average at the third institution, GRE verbal, GRE 
quantitative, TOEFL score, permanent city of residence, per- 
manent state, permanent zip code and graduate school action 
(R-reject, A-accept). 

Step 1: Difference Index 

The first level of analysis called the difference index was 
executed on the 17 applications that were accepted by the 
graduate committee. Applications Quest computes a differ- 
ence index for applications based on their overall difference. 
For example, the difference matrix in Table 1 contains a dif- 
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ference value for each pair of applications. The average of all 
the difference values for all the applications is the difference 
index. The difference index is a measure of the overall holis- 
tic diversity for a group of applications. The difference index 
for the 17 applications accepted by the graduate program 
committee was 44 percent. In other words the applications 
accepted by the graduate program committee were on aver- 
age 44 percent different or 56 percent similar, holistically 
speaking. Table 2 has some statistical data for the 17 ac- 
cepted applicants. 


Table 2. Average Scores for Accepted Applicants from 
Psychology Graduate Program Case Study 



Average 

Highest 

Lowest 

GRE Quantitative 

575 

710 

380 

GRE Verbal 

492 

630 

300 

INST1 GPA 

3.36 

4.0 

0.00 


This group consisted of 11 females and six males, 10 
doctoral applicants and seven masters, 11 U.S. citizens and 
six not specified. Along the race attribute there was one His- 
panic, three Asian, 11 white and two not specified. The next 
step in the study, involved purging applications that did not 
meet the department’s minimum requirements. 

Step 2: Qualified Applicant Pool 

There are exceptions in every admission group, i.e. those that 
are accepted below the minimum GRE and GPA requirements. 
In order to accommodate for this effect, all of the applicants 
that met or exceeded the lowest GPA and/or GRE score of the 
lowest accepted applicant were kept in the applicant pool. 
All those that did not meet this requirement were deleted 
from the pool. Those applicants that met this minimum re- 
quirement are referred to as the qualified applicant pool. The 
qualified applicant pool contains all the applicants that could 
be accepted based on the minimum qualifications of those 
that were actually accepted. Next, the qualified applicant 
pool was processed in Applications Quest. 

The qualified applicant pool contained 207 applications. 
The difference index for the qualified applicant pool was 41 
percent, compared to 44 percent for the accepted applicants. 
The average scores for the qualified applicant pool can be 
found in Table 3. 

The qualified applicant pool consisted of 156 females 
and 51 males, 193 doctoral applicants and 14 master’s, 135 
U.S. citizens, 8 foreign and 64 not specified. Along the race 
attribute there were three Native Americans, eight Hispan- 
ic, 10 Asian, 18 black, 152 white and 16 not specified. In 


the final step of this case study, the 207 applications were 
processed in Applications Quest and 17 applicants from the 
qualified applicant pool were recommended. 

Table 3. Average Scores for Qualified Applicant Pool From 

Psychology Program Case Study 



Average 

Highest 

Lowest 

GRE Quantitative 

582 

780 

260 

GRE Verbal 

503 

780 

300 

INST1 GPA 

3.08 

4.00 

0.00 


Step 3: Recommended Applicants 

The recommended and accepted applicants did not share any 
common applications, therefore none of the recommended 
applicants were the same as the accepted applicants. The 
difference index for the recommended applicant pool was 56 
percent versus 41 percent and 44 percent for the qualified 
applicant pool and the accepted applicants, respectively. As 
such, the recommended applicants were holistically 12 per- 
cent more diverse than the accepted applicants. Table 4 con- 
tains the average scores for the recommended applicants. No- 
tice that the average GRE scores for the accepted applicants 
(see Table 2) is very similar to those for the recommended 
applicants. The recommended applicants consisted of 12 fe- 
males and five males versus 11 and six, respectively, for the 
accepted applicants. There were 14 doctoral applicants and 
three master’s among the recommended applicants versus 
10 and seven, respectively for the accepted applicants. The 
program recommended seven U.S. citizens, four foreign and 
six not specified versus 11 U.S. citizens and six not specified 
for the accepted applicants. With respect to race, there was 
one Hispanic, four Asian, four black, six white and two not 
specified versus one Hispanic, three Asian, 11 white and two 
not specified for the accepted applicants. 


Table 4. Average Scores for Recommended Applicants 
From Psychology Program Case Study 



Average 

Highest 

Lowest 

GRE Quantitative 

580 

740 

360 

GRE Verbal 

455 

630 

300 

INST1 GPA 

2.72 

4.00 

0.00 


Summary 

“The U.S. Supreme Court ruling on Brown v. Board of Educa- 
tion (1954) monumentally changed forever educational op- 
portunities available to minority students. The Court unani- 
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mously voted segregated educational facilities were unequal 
and therefore, violated the equal protection clause of the 14 th 
Amendment of the Constitution. The Court mandated deseg- 
regation of all public schools in the country. It overturned 
the previous decision in Plessy v. Ferguson (1896) which 
permitted ‘separate but equal’ public facilities for minori- 
ties” (Garrison-Wade & Lewis, 2006) Now in the 21 st century, 
these policies are under attack. Recent court decisions have 
reached the conclusion that diversity is a worthwhile cause; 
as long as race, gender, national origin, etc. are not given 
preferential treatment (Gratz v. Bollinger, 2003; Grutter v. 
Bollinger, 2003). Current court cases are still being debated 
regarding race-conscious policies in school placement. Peo- 
ple, including the U.S. Supreme Court justices, are looking 
for solutions in this ongoing debate. We are still left with this 
question: How can one diversify without giving preference to 
race, gender, national origin, and/or any other attribute? 

The findings from a case study using Applications Quest 
suggest there are solutions using holistic review software. Ad- 
mission data for a graduate program were processed in Ap- 
plications Quest. The study revealed that Applications Quest 
produced holistically more diverse recommended applicants 
versus the graduate program committee’s accepted applicants 
with the same academic credentials. Specifically, the recom- 
mended applicants were holistically 12 percent more diverse 
than the accepted applicants. The recommended applicants 
were selected from a qualified applicant pool such that each 
applicant was equally qualified as the other applicants. The 
program grouped admission applications from a qualified ap- 
plicant pool such that the groups represented holistic, di- 
verse applicant pools where race, gender and national origin 
were one of many attributes used; no single attribute was 
the sole determinant of the recommendation. All application 
attributes were given equal consideration allowing consider- 
ation of race, gender, national origin, etc., without violating 
any of the aforementioned court decisions. 

Furthermore, the results produced are reproducible and 
measurable and can be integrated into any existing admis- 
sion process. This type of program can empower admission 
officers during the decision making towards a legal, fair and 
equitable admission process. The impact of this tool has 
not been observed, as of yet. However, it is clear that this 
program and others like it could be a “silver bullet” in this 
debate, such that future arguments for or against race-con- 
scious admission, school placement policies, and academic 
programs will be neutralized based on the current status of 
legal decisions. 


Recommendations for Selective Graduate, Profession- 
al and Undergraduate Admission 

Use a holistic admission program in a three phase ad- 
mission process as follows: 

1. Identify your diversity attributes (e.g., race, 
ethnicity, gender, political affiliation, 
religion, etc.) and require all applicants to 
submit diversity information. 

2. Identify applicants that are “Automatic 
Admits” and accept them. 

3. Identify the “Qualified Applicants” (those 
that meet your minimum requirements) and 
process them in software like Applications 
Quest to fill the remaining acceptance slots. 

4. Reject all other applicants. 

Recommendations for K-12 School Placements 

Use a holistic admission program as a diversity place- 
ment tool if the number of students preferring a spe- 
cific school exceeds the number of seats available at 
the school as follows: 

1. Identify your diversity attributes (e.g., race, 
ethnicity, gender, political affiliation, religion, 
etc.) and require all families to submit diver- 
sity information. 

2. Collect the top 3-5 school preferences for 
each child in addition to the diversity attri- 
butes. 

3. Pre-group the students by school preference. 
In other words, all students who select School 
A as the first, second, third, etc. choice and 
process them in the software by school pref- 
erence. 
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